Field
Embodiments relate to providing a three dimensional (3D) television experience to viewers. More particularly, embodiments relate to reformatting 3D television data into a two-dimensional (2D) television data format for distribution to viewers.
Background
Now that the 3D viewing experience has migrated from the theater to the home, 3D televisions are becoming increasingly popular. As a result, content providers face increasing demand for 3D content. This, in turn, requires broadcast service providers, such as terrestrial and non-terrestrial providers to develop techniques for distributing 3D content to their subscribers, the ultimate viewers.
Distribution of 3D video requires distribution of stereoscopic 3D images. A stereoscopic 3D image comprises one image for the left eye and one image for the right eye. At the receiving end, the left and right eye images are presented to the viewer's left and right eyes, respectively, to achieve the 3D effect.
If bandwidth were not a problem, the requisite left and right eye images could be sent in their entirety. Such transmission would be lossless. However, for distribution to the home, bandwidth and channel usage conservation are essential considerations. For example, sending independent full resolution stereoscopic left and right eyes images would require using two channels to deliver a single television program. Because of a need to conserve bandwidth, two channel delivery for a single television program is unsatisfactory.
To avoid the requirement that two channels of bandwidth be used to transmit a single 3D program, a 2D frame compatible 3D format was introduced. In a 2D frame compatible 3D format, the left and right eyes images are packed into a single 2D frame. To accomplish this, the left and right eye images are subsampled. The subsampled left and right eye images are then combined to create a 2D image, which can be stored in a conventional 2D format. While this combined image is not a real image in a traditional sense, the combined left and right eye images fit into the 2D frame. The 2D frame containing the subsampled left and right eye images is then distributed for processing and viewing. Such 2D frame compatible 3D is implemented by most major broadcast service providers, including broadcast systems over satellite and cable channels.
The required subsampling can be performed in a number of ways. For example, one subsampling technique involves subsampling the left and right eye images in the horizontal direction. In vertical subsampling, alternate rows of the left eye image and alternate rows of the right eyes images are selected to be combined. The combination involves placing the subsampled images (alternate rows of the left eye image and alternate rows of the right eye image) on top of one another in an over-under configuration to pack the 3D image in a format compatible with a 2D frame. Alternatively subsampled rows of the left and right eye images can be interleaved with one another to pack the 3D image in a format compatible with a 2D frame.
Another subsampling technique involves subsampling the left and right eye images in the vertical direction. In horizontal subsampling, alternate columns of the left eye image and alternate columns of the right eyes images are selected to be combined. The combination involves placing the subsampled images (alternate rows of the left eye image and alternate rows of the right eye image) adjacent to one another in a side-by-side configuration to pack the 3D image in a format compatible with a 2D frame. Alternatively subsampled columns of the left and right eye images can be interleaved with one another to pack the 3D image in a format compatible with a 2D frame.
This resulting 3D image is then compressed and transmitted to a set top box. For example, MEPG-2 or MPEG-4 coding are common compression techniques used. The set top box decompresses the received 2D frame, and transmits the decompressed 2D frame to a television. The television separates the packed left eye image and right eye image, and then creates independent left eye and right eye images in full screen resolution for display as a 3D image.
Another form of subsampling is known as checkerboard subsampling. Checkerboard subsampling involves selecting alternate pixels in row and columns of the left eye and right eye images. Theoretically, checkerboard subsampling provides optimal performance in the absence of compression. This is because with either vertical or horizontal subsampling, resolution in the direction of the skipped rows or columns is halved as filters are only applied in the direction of the subsampling to account for skipping rows (vertical subsampling) or skipping columns (horizontal subsampling).
With checkerboard subsampling, however, a 2D filter can be applied. As a result, the horizontal and vertical bandwidth of the video image can be increased. Although checkerboard subsampling results in a reduction in resolution in the diagonal direction, this is generally less noticeable. As a result, checkerboard subsampling better preserves horizontal and vertical frequencies.
One type of 2D filter used in checkerboard subsampling is a quincunx filter. In general, a quincunx filter passes higher frequencies in the horizontal and vertical directions, but lower frequencies in the diagonal direction. Checkerboard subsampling with quincunx filtering is considered optimal for preserving image quality, thereby providing a better video experience.
However, there is a significant problem with conventional checkerboard subsampling when the subsampled 3D data is packed into a 2D format for transmission as a 2D image, whether the data is packed in an over-under configuration or side-by-side configuration. The problem is that conventional packing formats for checkerboard data do not compress well. The problem manifests itself in that when conventional compression algorithms are applied to the packed checkerboard data, the resulting data exhibits numerous annoying artifacts. These artifacts result from the shift between the rows and columns of the checkerboard subsampled data. That is, during packing alternate lines, or alternate columns, having samples that are shifted with respect to one another by virtue of the checkerboard subsampling are placed next to one another, but without any shifts. Edges in an underlying image when processed in this manner become jagged. The jagged edges create associated high frequency image components that cause annoying artifacts when the image is compressed. As a result of this problem with checkerboard subsampling, even though it performs better than horizontal or vertical subsampling in the absence of compression, because the reality is that video is generally compressed for transmission and checkerboard subsampling performs suboptimally in the presence of such subsampling, checkerboard subsampling is not used for distributing 3D content to service provider subscribers. Thus, while checkerboard subsampling offers promise of a better 3D viewing experience, it is not used due to the annoying artifacts that are created using conventional packing paradigms to pack 3D data into 2D compatible formats in systems where there is significant compression.